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This paper analyzes stochastic networks consisting of finite ca¬ 
pacity nodes with different classes of requests which move according 
to some routing policy. The Markov processes describing these net¬ 
works do not, in general, have reversibility properties, so the explicit 
expression of their invariant distribution is not known. Kelly’s lim¬ 
iting regime is considered: the arrival rates of calls as well as the 
capacities of the nodes are proportional to a factor going to inhnity. 

It is proved that, in limit, the associated rescaled Markov process con¬ 
verges to a deterministic dynamical system with a unique equilibrium 
point characterized by a nonstandard fixed point equation. 


1. Introduction. In this paper, a new class of stochastic networks is in¬ 
troduced and analyzed. Their dynamics combine the key characteristics of 
the two main classes of queueing networks: loss networks and Jackson type 
networks. 

1. Each node of the network has finite capacity, so a request entering a 
saturated node is rejected, as in a loss network. 

2. Requests visit a subset of nodes along some (possibly) random route, as 
in Jackson or Kelly’s networks. 

This class of networks is motivated by the mathematical representation of 
cellular wireless networks. Such a network is a group of base stations covering 
some geographical area. The area where mobile users communicate with a 
base station is referred to as a cell. See Figure 1. A base station is responsible 
for the bandwidth management concerning mobiles in its cell. New calls are 
initiated in cells and calls are handed over (transferred) to the corresponding 
neighboring cell when mobiles move through the network. A new or a handoff 
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Fig. 1. The motion of a mobile among the cells of the network. 

call is accepted if there is available bandwidth in the cell, otherwise it is 
rejected. 

Previously, these networks have been modeled at a macroscopic level as 
loss networks characterized by call arrival rates, mean call lengths, handoff 
rates and capacity restrictions on the number of calls, in the case of exponen¬ 
tial times. One of the main quantities of interest is the stationary blocking 
probability of the network at each node, defined as the stationary probability 
that a call arriving at that node cannot be accepted. Approximations have 
been used to analyze these networks; see [1, 5, 11] and the references therein. 

Assuming Poisson arrivals and exponentially distributed random vari¬ 
ables, the evolution of such a network with N nodes can be represented as 
a Markov jump process (Ai(t)) with values in some finite (but large) set S. 

It turns out that, contrary to uncontrolled loss networks with fixed routing, 
the Markov process {X{t)) is not in general reversible or quasi-reversible. 
Consequently, contrary to Jackson networks and the like, or uncontrolled 
loss networks, these networks do not have a stationary distribution with a 
product form. 

In this paper, the time evolution of these networks is analyzed by consider¬ 
ing Kelly’s scaling. The arrival rates and capacities at nodes are proportional 
to some factor N which becomes large. This scaling has been introduced by 
Kelly [8] to study the invariant distribution of loss networks. A study of 
the time evolution of loss networks under this scaling has been carried out 
by Hunt and Kurtz [6]. See [7] for a survey of these questions. A different 
scaling is considered in [2]. 

The equilibrium points. The time evolution of the network can be (roughly) 
described as follows. A stochastic process (Ajv(t)) associated with the state 
of the network for the parameter N is introduced: Ajv(t) is the vector de¬ 
scribing the numbers of customers of different classes at the nodes of the 
network. The equation of evolution for the network is 

— XNit) = FN{XNit)) + Misfit), t>0, 
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where {M]\[{t)) is a martingale which vanishes as N becomes large, is 
a somewhat complicated functional (associated with the generator of the 
corresponding Markov process) converging to some limit F. As N goes to 
infinity, it is proved that {X]\f{t)) converges to some function (x(t)) satisfying 
the deterministic equation 

(1) j^x{t) = F{x{t)), t>0. 

The equilibrium points of the limiting process are the solutions x of the 
equation F{x) = 0. It is shown in this paper (and this is a difficult point) 
that there is only one equilibrium point in Kelly’s limiting regime. 

Related work. For classical uncontrolled loss networks, the invariant prob¬ 
ability has a product form representation. Nevertheless, the evolution of 
these networks under Kelly’s scaling turns out to be quite intricate. Hunt 
and Kurtz [6] showed that at any x, the vector field F{x) driving the limit¬ 
ing dynamical system is related to some reflected random walk in with 
jump rates depending on x. Intuitively, the situation can be described as 
follows. At points x at which this random walk is ergodic, F{x) is expressed 
in terms of its invariant distribution; at x at which the random walk is 
transient, the exit paths to infinity determine F{x). It is not known, in gen¬ 
eral, whether there always exists a unique limiting dynamical system. Hunt 
and Kurtz [6], Bean, Gibbens and Zachary [3, 4] and Zachary [12] analyzed 
several examples with one or two nodes where uniqueness is shown to hold. 

Results of the paper. Using the terminology of cellular networks, users 
arriving in the network correspond to new requests for a connection in a 
cell. Different classes of customers access the network—classes differ by their 
arrival rate, by their dwell time at the nodes (i.e., the amount of time that a 
mobile of an ongoing call remains in a given cell), by their call duration and 
also by their routing through the network. During a call, a user moves from 
one cell to another according to some Markovian mechanism, depending on 
his class. When a user moves to another cell (node), this cell has to be 
nonsaturated to accommodate the user, otherwise the user is rejected (the 
call is lost). If it is not rejected during the travel through the network, the 
user call terminates after the call duration time has elapsed. 

For the networks analyzed in this paper, the uniqueness of the limiting 
dynamical system is not difficult to establish. The main difficulty lies in 
the complexity of the system of equations defining the equilibrium points 
of the dynamical system. Since there does not seem to exist some reason¬ 
ably simple contracting scheme to solve these equations, the uniqueness of 
the equilibrium points is therefore a quite challenging problem. For exam¬ 
ple, in Section 4.2, the case of a very simple network with two nodes and 
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two deterministic routes is investigated and the explicit representation of 
the equilibrium point is obtained, expressed in terms of quite complicated 
polynomial expressions involving the parameters. 

The paper is organized as follows. Section 2 introduces the Markovian 
description of these networks, Section 3 gives the convergence results, to¬ 
gether with the description of the limiting dynamical system. Section 4 is 
devoted to the main results of the paper—it is shown that, in limit, there 
exists a unique stable point for the network. The ingredients used to obtain 
this uniqueness result are: 

• a dual approach to the problem of uniqueness, that is, finding the set of 
parameters such that a given point is an equilibrium point of the dynam¬ 
ical system; 

• a key inequality proved in the Appendix; 

• a convenient probabilistic representation of a set of linear equations. 

The inequality proved in the Appendix involves a quantity related to relative 
entropy, but, curiously, it does not seem to be a consequence of a standard 
convex inequality as is usually the case in this type of situation. 

2. The stochastic model. The network consists of a finite set I of nodes, 
node i € I having capacity [ciN\, where Cj > 0 and £ N. This network 
receives a hnite number of classes of customers, indexed by a finite set R] 
class r € R customers enter the network according to a Poisson process with 
rate XrN, where > 0. 

• Call duration. A class r customer who thus far has neither been rejected 
nor routed to the outside (see Routing below) leaves the network after an 
exponentially distributed time with rate (call duration in the context 
of a cellular network). The case /ir- = 0 is not excluded; it corresponds to 
the situation where customers stay in the network as long as they are not 
rejected or routed to the outside. 
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Fig. 2. A network with two classes of customers. 
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• Dwell time. The residence time of a customer of class r at any node i € I 
is exponentially distributed with parameter 7 ^. Such a customer can leave 
the node before the end of his dwell time (due to the end of call) at rate 
f^r • 

• Routing. A class r customer entering the network arrives at some ran¬ 
dom node in I whose distribution is Qr and then moves from one node 
to another, or to the outside (referred to as node 0 ), according to some 
transition matrix on / x JU {0}. By changing the parameter of the res¬ 
idence time, it can be assumed without loss of generality that the matrix 

is 0 on the diagonal. 

• Capacity requirements. All customers require one unit of capacity at each 
node. 

All random variables used for arrivals, residence times and dwell times are 
assumed to be independent. 

This class of networks includes the case of classes of customers with de¬ 
terministic routing, as in Kelly’s networks, and also classes of customers 
with Markovian routing, as in Jackson networks. Figure 2 represents a net¬ 
work with two classes of customers—class 1 customers follow a deterministic 
route, while class 2 customers can reach either node 1 or node 3 from node 
4, and the capacities of the nodes are 5. Note that for the general model, no 
assumptions have been made concerning the transition matrices •)) so 

some classes of customers may achieve inhnite loops in the network. 

Notation. For i ^ I, r ^ R and t > 0, Xl^.^{t) denotes the number of class 
r customers at node i at time t. {X^{t)) = {Xl^j.{t),i ^ I,r ^ R) is the cor¬ 
responding process. The renormalized process is defined as 

X^it) = ^X.^rit) 
and X^{t) = (Xf^^{t),i€ I,r€R). 

Denote hy R C I the set of nodes which can be visited by a class r 
customer, that is, i € R when i is visited with positive probability by the 
Markov chain with transition matrix p^^'^ and initial distribution It is 
assumed that I = state space of the Markov process {X^{t)) 

is 

S]\f = |x = (xi^r) £ : ^2 ^i,r ^ CjA^ and Xi^r = 0 i ^ R 

The Q-matrix (Ajv(a:,y)) of {X^{t)) is given follows: 

External arrival of a class r customer at node i: 

An{x, X + Ci^r) = XrNqr{i)t{x+ei^,&SNA 
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Service completion, rejection by a cell or a transition to the outside: 

An{x, X - Ci^r) = Xi^r {^r + {i, + 7rP^''^ {h 0)^ ; 

Transfer from node i to node j: 

An{x, X - ei,r + ej,r) = 

where Ci^r is the unit vector at coordinate (i,r). The state space of the 
renormalized process (X^(t)) is given by 

Tc = = (xi^r) £ ^ Xi^r < Ci and Xi^r = 0 if i ^ |, 

the subscript c= (ci) of Xc standing for the vector of capacities. 

3. Convergence resnlts. The following proposition establishes the deter¬ 
ministic behavior of X^{t) as N goes to infinity. This result is the conse¬ 
quence of the fact that the stochastic perturbations of the original system 
are of order y/N and therefore vanish because of the scaling in 1/N. 

To describe the time evolution of the network, one introduces the follow¬ 
ing Poisson processes: denotes a Poisson process with parameter ^ > 0, 

and an upper index p G d G N, is added when several such Poisson 
processes are required. For example, for i G / and r G R, A/A^g,,(i) is the ex¬ 
ternal arrival Poisson process of class r customers at node z. In a similar 
way, for /c > 1, A/"^ ^ is the Poisson process associated with the transfer 

of the /cth class r customers from node z to j G / U {0}. 

For t>0 and (z,r) G I x R, let Y/^it) = [ciN\ — ^be quantity 

YJ^{t) is the size of the free space at node z. The process {X^(t)) can then 
be represented as the solution of the following stochastic integral equation: 

+ / '^{Yf{s-)>0}-^XrNqr('i){ds) 

0 

- Y YJ ^{k<xi^As-)}^Yr)^ij){ds) 
ie7u{o}fc>i ^ 
j¥=i 


(2) 
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Here f{t—) denotes the limit on the left of the function f at t. By com¬ 
pensating the Poisson processes, that is, by replacing the differential term 
M^{ds) by the martingale increment Af^{ds) — ^ds, one gets the identity 

+ XrNqrii) f ds 

0 

j&i "'o 

-ilr + Pr) [ Xll^{s-)ds, 

JO 

where (M/^(f)) is the martingale obtained from the compensated integrals 
of the previous expression. 

Denoting the renormalized martingale one finally 

gets 

X + ^rqr{i) f l{yV 

^ 0 

( 4 ) [ Xf^r{s-)^{Y.^(s-)>0}ds 

jei "'o 

pt _ 

(T^’ X Hr') I ) ds. 

JO 

The evolution equations for the renormalized process having now been writ¬ 
ten, one can establish the main convergence result: 


Theorem 1. If the initial state X^ (0) converges to x £ Xc as N goes 
to infinity, then {X^{t)) converges in the Skorohod topology to the solution 
{x{t)) of the following differential equation: For {i,r) G IxR, 


(5) = (^Kqr{i) X 'yrYlxj^r{t)p^''\j,i)^Ti{x{t)) - {jr X Hr)Xi,r{t) 


with x(0) = X and 

1 ; if ^ ^ ^i,r ^ Q? 

r 

A 1, otherwise, 
where a Ab = min(a, b) for a, b gM. and 


rfix) = 


i de/. 

Px = 




Er [^r<?r (0 + 7r J2j (j, i)] 
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By convergence in the Skorohod topology, one means convergence in distri¬ 
bution for the Skorohod topology on the space of trajectories. 

Proof of Theorem 1. Recall that if and two independent 

Poisson processes, and if Mp{t) = A/’^p((0, t]) — ^pt, p = 1, 2, are their associ¬ 
ated martingales, the latter are orthogonal in the sense that {Mi[t)M 2 {t)) 
is a martingale, that is, the bracket process {M\,M 2 ){t) is 0 for all t > 0; 
see [10]. The same property holds for stochastic integrals of previsible pro¬ 
cesses and {H 2 {t ))—for t > 0, 



The increasing process of the renormalized martingale defined above is 



and the increasing process in the right-hand side of the last equation can 
be evaluated by using the orthogonality of independent Poisson processes 
mentioned above. By using the fact that, for (i, r) G / x i? and t>0, < 

[ciN \, one obtains that there exists some constant K such that 


E([Mf,(t)]2) = E((Mf„Mf,)(t)) < KNt. 


Doob’s inequality implies that the martingale converges a.s. to 0 

uniformly on compact sets. Hence the stochastic fluctuations represented by 
the martingales vanish in limit. 

Now, by using the results of Kurtz [9], similarly to their use in Hunt and 
Kurtz [6] for loss networks, one can prove that any weak limit X = {Xi^r) of 
the process X^ satisfies the following equations: For (f,r) G / x i?, 



( 6 ) 



where = {m = {rrij) G (NU {-|-oo})^ : nr, > 1} and for x = {xir) G Xc, tTx is 
some stationary probability measure on = (N U {-poo})^ of the Markov 
jump process whose Q-matrix {Bx{-,-)) is defined as 


Bx{m,m — Ci) =''^Xrqr{i) if ruj > 1 


r 


Bx{m,m + ei) = ^ + 7r- P*''’^(b0) -P 



Bx{m,m- ei + Cj) = ^^rXj,rP^''\j,i) if m* > 1 


r 
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where e* denotes the ith unit vector of Moreover, the probability distri¬ 
bution Tlx has to satisfy the following condition: 

(7) 7r3;(m G : mi =-Loo) = 1 if Xj^r < Q- 


The Markov process (m^(t)) associated with the matrix Bx{-, ■) describes the 
evolution of {t/N) = {t/N)), that is, the time-rescaled process of the 

numbers of free units of capacity at different nodes during a time interval 
[t,t + Ndt[ when the renormalized process is around x on the normal 
time scale. Compared to {X^{t)), the process (Y^{t)) indeed evolves on a 
rapid time scale, so that 

_ 

l{y,^(s-)>o} ds ~ 7:x{'^i)dt, 

that is, such quantities can be replaced, in limit, by the average values of 
indicator functions under some limiting regime tTx of Y^ when X^{t) ~ 
X. Hunt and Kurtz [6] provide a detailed treatment of these interesting 
questions; see also [3, 4] and [12] for the analysis of several examples. 

In our case, the marginals of {rrY{t)) are also Markov, due to the fact 
that each customer occupies only one node at a time so that acceptance at 
node i only depends on the number of free units there. For i G /, the process 
of the number of free units at node i when the renormalized process 
is around x is a classical birth and death process on N whose rates are given 
by 


q{m,m + 1 ) = -|- Hr)xi^ri 

r 

q{m,m- 1) = if m > 1. 

The point -Loo is an absorbing point. Under the condition 
(8) “L Pr)Xi^r < ^ ^ 'Jr ^ ^ Xj^rP^ ^ (j) j ; 

r r \ j / 


the geometric distribution with parameter 


^{ir + Pr)Xi,r / ( ^rQrii) + 7r- 0 j = PI 

r r \ j ) 


and (5+00) tbe Dirac distribution at -Loo, are the two extreme invariant mea¬ 
sures of the process {rrY{t)). If ~ condition (8) holds, then 

the quantity 7r3;(N() is necessarily some convex combination of 1 and 
For such an z G /, by summing equations (6) over r, it is easy to check that 
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the quantity 7r3;(Nf) cannot be more than p^. (otherwise the hnite capacity 
condition J2r^i,r < Q would be violated). One gets that 7r2,(Nf) = 

The other cases follow from condition (7) or the transience of the process 
Since the differential equation (5) clearly has a unique solution, the 
theorem is proved. □ 

Remark. For t > 0, the above proof shows that the quantity Ti{x{t)) 
can be interpreted as the probability that a call is accepted at node i at 
time t. If the limiting dynamical system has a unique equilibrium point x 
(which will be shown in the sequel), then by using arguments similar to these 
in [6], Ti{x) can be seen as the limiting stationary probability that a call is 
accepted at node i. 


4. Equilibrium points. Theorem 1 shows that equilibrium points x ^ Xc 
of the limiting dynamical system, that is, those x that satisfy x[^{t) = 0 
for any (i,r) £ I x R and t>0 when (xj^r.(0)) = x, are the solutions of the 
following set of equations: 

(9) {'Jr + fJ-r)Xi,r = (^Kqr{i) +lrY^Xj^rP^''\j,i)^Ti{x), {i,r) £ I X R, 

where Ti{x) is defined as in Theorem 1. Note that Ti{x) £ (0,1] and that 
either Ti{x) = 1 or = Q- 


4.1. Characterizations and existence of equilibrium points, li x £ Xc sat¬ 
isfies (9), it is a solution of the equations 

(10) (Tj. + Pr'jXi^r — T Tr ^ ^ ^j,rP^ ^ (T ^)^ li (*) r'j £ I X 7?, 

for some t = (tj) G (0,1]^ such that for any i£ I, either t j = 1 or Yl,r ^i,r = Cj. 

Conversely, if x G Tc is a solution of (10) for a fixed i £ I, then there are 
two cases: 


• If XrQrii) + 7r J2j i) = 0 for all r £ R, then Xi^r = 0 for all r and 

thus, necessarily, rj(x) = 1 and equations (9) hold trivially. 

• Otherwise, by summing these relations over r £ R, one gets the identity 

'Eri^^rQrii) +1rT,jXj,rP^^Hj,i))' 

If tj = 1, then = 1 and so, by definition of Ti{x), Ti{x) = 1 = tj. If < 1, 
then due to the above assumption, we necessarily have J2r^i,r = Ci, so 


Ti{x) 


_ T Pr)Xi^r _ ^ ^ 

Y.riKqrii) + 7rY.j Xj,rP^^Kj^i)) 


Equations (9) are thus satisfied for x. 
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The following characterization of equilibrium points of the system has thus 
been obtained: 


Proposition 1 (Characterization of equilibrium points). The equilib¬ 
rium points of the limiting dynamieal system are the elements x € Xc such 
that there exists some t G (0,1]^ satisfying: 

1. For any {i,r) £ I x R, 

(11) Xj,r= (^arqrii) + PrY^Xj^rP^'^\j,i)^ti. 

2. For any i£ I, either ti = l or Xi,r = Ci, 

where Or = K/ [ir + hr) 0,nd f5r = 7r/(7r + hr) for r £ R. 

To prove the existence of a fixed point, a second characterization of equi¬ 
librium points will be useful: 


Proposition 2 (Existence of equilibrium points). The equilibrium points 
of the dynamical system (5) of Theorem 1 are the fixed points in Xc of the 
funetion <hc defined by, for x £ Xc, 


(12) ^'c(x) 



where, for z> 0 and u £ [0, +oo)^, 

Qz{u) = f A l)u. 

The funetion has at least one fixed point. 


Proof. Note that the function ©c maps [0, +oo)^ into the subset {u £ 
[0, +oo)^ : X)r '0‘r < c} and ‘hc(x) indeed belongs to Xc- its (i, r)th coordinate 
is 0 whenever i ^ R. 

The characterization of equilibrium points follows from Proposition 1 and 
by noting that, for u £ [0, +oo)^, z > 0 and v £ [0, +oo)^ such that J2r ^r < 
z, there is an equivalence between the identity Qz{u) = v and the fact that 
there exists some t £ (0,1] such that v = tu and either t = 1 or J2r Vr = z. 

The existence of a fixed point is then a consequence of Brouwer’s fixed 
point theorem, since Xc is a convex compact subset of and is a 

continuous function from Xc into itself. □ 
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4.2. The example of deterministie routes. Requests of class r use a deter¬ 
ministic route of length L G N U {+00} consisting of a sequence /r = (V) 0 < 
p < L) with values in I such that 

qr{io) = l, p^'^\ip,ip+i) = l forO<p<L-l 

and 0) = 1 if L < -|-oo. Note that, since I is finite, the case L = -|-oo 

necessarily corresponds to a route r which eventually becomes periodic. 
Equilibrium points as described in Proposition 1 can be written more ex¬ 
plicitly in terms of t solving (11) 

as follows: 


1. For a nonperiodic deterministic route, L < -|-oo, these equations reduce 
to a recursion—for 0<p<L, 


p 

Xip,r = OirPr . 

k=0 

2. For a periodic route r consisting of nodes io, ii,..., ifc-i and then the 

infinite loop ik,ik+i, ■ ■ ■ ,ik+i-i,ik,ik+i, _ Provided that the are 

such that < 1, the solutions are given by 


(13) 




Xih,r — 


= arPr t 


'^ra 1 


0<m<h 
^h4 


Oir(^r ^20 ‘ 

1 -...t 




0 < /i < A: — 1, 


h>k. 


The above calculations show that an equilibrium point {xi^r) has a polyno¬ 
mial expression in t = {tj) whose degree is related to the rank of i along the 
route in the case of a nonperiodic route, and that (xi^r) is given by a power 
series in t when the route r is periodic. Moreover, these quantities have to 
satisfy the following constraints: for i G I, then either tj = 1 or J2r ^i,r = Ci- 
The exact expression of fixed points in the case of deterministic routes is 
therefore very likely to be nontractable. As will be seen, even the uniqueness 
is not a simple problem. 

The complexity of exact expressions is illustrated by a simple example 
of a network with two nodes, I = {1,2}, and two deterministic nonperiodic 
routes: the first class enters at node 1, goes to node 2 then exits, whereas 
the second class does the opposite. Take = /i2 = 0 so that /3i = /?2 = 1- It 
is then easy to show that: 



ANALYSIS OF LOSS NETWORKS WITH ROUTING 


13 


1 . An equilibrium point associated to (ti,t2) with = t2 = 1 exists if and 
only if ai 02 < ci and ai + a2 < C2. In this case, xiq = X2,i = ai and 
®1,2 = X2,2 = ct2- 

2 . An equilibrium point exists with ti = 1 and t2 < 1 if and only if 

02 


Ol + 


C2 < Cl and ai + 0:2 > C2. 


Ol + Q!2 

Under these conditions it is then unique: 

2:1,1 = Ol, 2:2,1 = Ol 
C2 


C2 


Ol + 0:2 


Ti ,2 — 2:2,2 — 0:2- 

Oi + O 2 

3. By symmetry, analogous results hold with U = 1 and t2 < 1- 

4. An equilibrium point exists with ti < 1 and t2 < 1 if and only if 

O 2 Ol 

ai H-C2 > Cl and 02 ^ -ci>C2. 

0:1 + O 2 Ol + 02 

In this case the solution is unique: 

xi^i=oiti, 2:2,1 = Q;itit2) 

Xl^2 = Oi2tlt2, 2:2,2 = 0:2^2) 

with 

(oici — q:2C2 — 0:10:2) + (oici — a2C2 — 0102 )^ + 4ci02of 


t^ = 


2 o? 


t2 having a similar expression with the subscripts 1 and 2 exchanged. 

It is not difficult to check that these four cases are disjoint and cover all 
situations. Therefore, the uniqueness of the equilibrium point holds in this 
case. 

A similar approach does not seem possible for a more complicated sys¬ 
tem of deterministic routes. Even proving uniqueness in such a context is 
challenging. 

4 . 3 . Uniqueness of equilibrium points. In view of Proposition 2 , to prove 
the uniqueness of equilibrium points, a contraction property of <I>c would 
suffice. But it can be shown that ‘he is generally not a contraction for classical 
norms. 

For example, in the simple network considered above with / 3 i = /32 = 1 , 
the equation <I>c(x) = y is 

(yi,i, 2/1,2) = ©Cl (ai, 2:2,2) and (2/2,1,2/2,2) = 0c2(2:i,i,a2)- 
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When Cl > ai and C 2 > 0 : 2 , one can choose x and x' in Xc such that 

CD + 3:2,2 < Cl, ai+3:2,2 < Cl, 3:1,1 +02 <C 2 , 

3 :'i,i +02 <C 2 , a:i ,2 = xi, 2 , X 2 ,l= 3:2,1- 

Then, in this case, ||<hc(3:) — <hc(3:0llp = l|3: — x'\\p for p G [1, + 00 ], where ||x||p 
is the Lp-norm (||3:||p)^' = |3:i,r|^ for p < +00 and ||x||oo = niax{|xj,r| : 

(i,r) G / X i?}. 

Under the condition max{/3j,; r G R} < 1 and in the case of deterministic 
nonperiodic routes, the function 


a. 


.qrii) + PrY^Xj^rP^''\j,i),{i,r) €l X Rj 
i / 


is a contraction for any Lp-norm. However, the same property does not 
necessarily hold for <hc, since it can be shown that the function 0c, c > 0, 
is not a contraction for any Lp-norm on [0,+ 00 )'^, except when |i?| = 1, or 
when |i?| = 2 and p = + 00 . 


A dual approach. To prove uniqueness in the general case, the point 
of view is changed—instead of looking for x G Tic which are equilibrium 
points of the limiting dynamics associated to a given vector c= (cj,z G /) G 
(0, + 00 )^ of capacities, an element x is given and one looks for the set of 
vectors c such that x is a equilibrium point of the limiting dynamics. The 
uniqueness of the equilibrium point for a given c is then equivalent to the 
property that those sets of vectors associated to two different values of x do 
not intersect. 

Define 


Tloo '^=’ {3: G [0, + 00 )“'^'“ :xj,c = 0 if i ^ R}- 

It is of course enough to consider the solutions x in X^o that satisfy (11) 
for some t G (0,1]“^. The first step of this analysis is to show that for any 
t G (0,1]^, a solution x to the system of equations (11) is at most unique. 

Proposition 3 (Probabilistic representation), //t G (0,1]'^ is such that 
the system of equations (11) has a solution in X^o, this solution is unique 
and can be expressed as 


' +00 


(14) 


hr - ^r^ ( XI n 


, A:=0 p=0 


y {i,r) G I X R, 


where (Zn^) is a fpossibly killed) Markov chain with transition matrixp^'^\-, •) 
and initial distribution q^. 
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Note that the above expression for {xi^r) generalizes the formula obtained 
for periodic deterministic Markovian routes since, using the same notation 
as in the example of periodic deterministic routes, equation (14) gives, for 
h>k, 


+ 00 


^ih — (y-rPr 


j=0 


■t 


^k+l-H 


V - 


UrPrti ■■■t 


'^h 


1 


which is formula (13), and = arfi^ti^ " ''^ih h < k. 


Proof of Proposition 3. The system of equations (11) splits into 
|-R| subsystems of equations, one for each r € R, with unknown variables 
{xi^r, i £ Ir)- Consider just one of these |ii| systems and remove the index 
r for simplicity. J is defined as the range in I of the Markov chain [Zk) 
with initial distribution q and transition matrix p(-, •). Such a subsystem of 
equations can be expressed as 

Xi = ^ag(i) + Xjp{j, i)^ U, i G J. 

This system of equations has a solution since the system of equations (11) 
is assumed to have one. For i G J, set y* = Xi/{ati) (remember that both a 
and ti are positive). The vector y = {yi) then solves the equations 

(15) yi = qii)+^yjP{j,i), 

j 

with P{j, i) = (5tjp{j, i). The matrix P = {P{i,j)) is sub-Markovian and (Zn) 
denotes the Markov chain with initial distribution {q{i)) and transition ma¬ 
trix P. For f G J, clearly yi > q{i) = P(.^o = *)) and by induction, the above 
equation gives that, for n > 1, 

^ {Zo=l} {Zl=l} {Zn=l}^ 

By letting n go to infinity, we get 

!/<>». = E(gij ) Vier 

\fc =0 / 

For any i€ J, the above inequality implies that 

+ 00 

^P(z,z) < +00, 

k=0 

leading to the conclusion that the state i is transient for the Markov chain 
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It is easy to check that (ui) is also a solution of (15). Consequently, the 
nonnegative vector (vi) = {yi — Ui) satisfies the equation 

= ieJ, 

j 

which is the invariant measure equation for this Markov chain. Since all the 
states are transient, we necessarily have u* = 0 for all i € J. The uniqueness 
is thus proved. It is easy to check that the representation of (xj) in terms of 
the Markov chain (Z^) is indeed given by the representation of (uj) in terms 
of the Markov chain {Zn)- The proposition is thus proved. □ 

Definition 1. The set T is the subset of t e (0,1]^ such that the system 
of equations (11) has a solution, denoted by x* = (x*^) (it is unique by the 
above proposition). For t G T and i G /, define 

C +OO k 
fc=0 p=0 ^ 

where (Zn^) is, as before, a Markov chain with transition matrix 
and initial distribution qj.. 


Lemma 1 (Strong monotonicity). If t = {ti) and t' = (t() are elements 
of T sueh that, for any i £ I, 

ti<t'i cri{t) > ai{t') and t'i<ti cri{t') >ai{f), 

then t = t'. 


Proof. The assumption on t and t' gives 

(16) ^\ og { t ' Jti ){ ai { t ') - ai { t )) < 0. 

iei 


The definition of a,- gives the following representation for the difference 


cfi{t') - ai{t) = y^arlE 

r 




Lfc=o 





Note that, as in the proof of Proposition 3, the inhnite sums within the 
expectation are integrable, thereby allowing these algebraic operations. By 
substituting this expression into (16) and exchanging summations first on 
i G / and r £ R and then on i G I and fc G N (remembering that I and R are 
hnite), one gets 


Or-IE 



n n ^2 


.(r) 


h=0 


h=0 




<0 
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and, by extending the definitions of t and t' to the coordinate 0 so that 

to = to = 1, 



Proposition A.l of the Appendix applied to the expression inside the ex¬ 
pectation implies that, with probability 1, this integrand should be 0. Con¬ 
sequently, the same proposition implies that for any r G R, the identity 
t (r) = t' holds almost surely for any k gN. Hence, U = t' for any i G R 

and any r G Rhy definition of R- One concludes that t = t', since I = [jrR- 
The lemma is thus proved. □ 

The main result concerning the equilibrium points of the limiting dynam¬ 
ical system (5) can now be established: 

Theorem 2 (Uniqueness of equilibrium points). There is a unique equi¬ 
librium point of the dynamical system {i,r) G I x R) defined by (5). 

Proof. For t gT, define Ct as the set of vectors c = (cj) G (0, -|-oo[^ such 
that X* is a hxed point of the dynamical system associated with capacities 
(cj). For t gT and c G (0, -|-oo)^. Proposition 1 shows that if c G C* then, for 
any i G I, afit) < Ci, and when U <1 then afit) = Ci. 

For t, t' gT, assume that there exists some c G Ct H Cf ■ If i G /, the re¬ 
lation ti < implies that U <1 and therefore that afit') < a fit) = c*. From 
Lemma 1, one concludes that, necessarily, t = t'. The uniqueness of equilib¬ 
rium points readily follows from the result that if z and z' are equilibrium 
points of the dynamical system (5) associated with some vector of capaci¬ 
ties c G (0, -|-oo)^, then there exist t and t' gT such that z = x^ and z' = x^ . 
Since c G Cj H Ct', we have t = t' and therefore z = z'. The theorem is thus 
proved. □ 


APPENDIX 

This section is devoted to the proof of a key technical result for the proof 
of the uniqueness of equilibrium points. It involves an expression which bears 
some similarity to a relative entropy. 

Proposition A.l. Let u = (uj)jgN o,nd u' = (u')jgN two sequences of 
elements of (0,1]. If the series 

^ log(u7ui) f n ^ “ n 

converges, then its sum is nonnegative and equals 0 if and only if u = u'. 
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Proof. It is first proved by induction on n G N that for any u, u' G 

(o,ir, 


(A.l) 


fniu,u') = J2^0giUi/Ui) f Wu'j - Y[uj \ > 0. 

2 = 0 ' 


This is obviously true for n = 0. Now assume this inequality holds for any 
integer k <n. Let u and u' be some hxed elements of (0,1]"'. 

• If there exists some k such that 1 < A: < n and 


n n 


u-i 


- n ^ 

j<k 


j < 0, 


^j‘^k—1 j'^k—l 
then fn{u,u') can be decomposed as follows: 

fn{u,u') = fk-l{{uo, Uk-l), ( m ' o , . . 


(A.2) 


+ fn-k 


\j<k 


, Uri 


\j<k 


-log Y[ U, 
\ j<k—l 


n 

i<k-l 


im - n '^1 


^j<k 


j<k 


.,Ur 


From the induction hypothesis and the assumption on k, all terms of the 
right-hand side of this identity are nonnegative, so fn{u,u') > 0. 
Otherwise, for any 0 < k <n, the quantity IljXfc “ IljXfe '^j a con¬ 
stant sign and is not 0 (positive, say). There are two cases: 

1. If tifc < for all k such that 0 < A: < n, all terms in the sum defining 
fn{u,u') are nonnegative, and hence fn{u,u') > 0. 

2. If not, let A: < n be the first index such that Uk > u'^- Since uq <Uq, we 
have A: > 1 and can write 


fn{u,U ) — fn—1 [('^0) • • • ) 2) '^k—l'^ki • • • j Un')i 

(u'o,..., <_2, <_1<, . . . , <)] 

+iog«_i/ufc-i) (1-4) n uj-(i-uk) n 

^ j^k—1 j^k—1 

The first term is nonnegative from the induction hypothesis. The sec¬ 
ond one is also nonnegative, since Uk-i < u'j^ < and IljXfc-i ^ 
nj<fc-i4- Therefore, fn{u,u') > 0 also holds in this case. The proof 
by induction is thus completed. 
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Inequality (A.l) is thus true for any n G N, implying that for any u, u' G 

(o,ir, 

foc{u,u') log{u[/ui) f Wu'j - > 0 

2=0 j'^'i ' 

whenever the series converges. The first part of the proposition is thus 
proved. 

Assume now that fooiu,u') = 0 for some u, u' G (0,1]^ such that the 
series converges. Using the same kind of decomposition as in equation (A.2), 
foo{u,u') can be expressed as, for some hxed A: > 1, 

foo{u,u') = fk-l{{uo, . . .,Uk-l), {Uq, 


+ /oo 




- log f n n f im - n = o- 

The second term of the right-hand side is clearly well defined, since foo{u, u') 
is. The first and second terms being nonnegative, we have 

iog[ n n “i) (im - n ^ o. 

Consequently, the difference u'qu'i ■ ■ - — uqUi ■ ■ - Uk has a constant sign for 

any /c G N. It can be assumed that these expressions are nonnegative. 

1. If rtj < holds for any i > 0, then each term of the infinite sum defining 
foo{u,u') is nonnegative and therefore null, since fcxi{u,u') = 0. It clearly 
implies that Ui = for all i G N. 

2. Otherwise, since uq < u'q, define n > 1 as the smallest integer such that 
Un > u'^- Since uqUi ■ ■ - Un < u'qu'i ■ ■ -u'^, there exists some index i < n 
satisfying Ui <u[. Dehne k as the largest such index. In particular, for 
k <i <n, one has Ui = u[. Therefore, 

/oo ^ ) — /oo I ( ^0) • • • ) ^fc—1) Uj , Un^l ) • ■ • 

V V j=k 

n 

^0) • • • ) '^k—li ^ji '^n+li ■ ■ ■ 
j=k 

+^og{u'k/uk)iU- n n 

\ V k<j<n / j<k \ k<j<n / j<k / 



= 0. 
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The first term is nonnegative and it is easily checked by using the defini¬ 
tions of k and n that the second one is positive. This equality is therefore 
absurd. This second case is not possible. 

The proposition is thus proved. □ 
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